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Abstract 

Tele-existence is an advanced type of teleoperation system that 
enables a human operator at the controls to perform remote manipulation 
tasks dexterously with the feeling that he or she exists in the remote 
anthropomorphic robot in the remote environment. In this report the concept 
of the tele-existence is shown, the principle of the tele-existence display 
method is explained, some of the prototype systems are described, and its 
space application is discussed. 

1. Introduction 

Tele-existence aims at a natural and efficient remote control of 
robots by providing an operator a real time sensation of presence. It is an 
advanced type of teleoperation system that enables a human operator at the 
controls to perform remote manipulation tasks dexterously with the feeling 
that he or she exists in one of the remote anthropomorphic robots in the 
several remote environments [1,2]. Similar concept is called artificial 
reality [3] or telepresence [4]. 

Fundamental studies for the realization of the tele-existence 
system are now being conducted in the authors' division of the Mechanical 
Engineering Laboratory (MEL) as part of the National Large Scale Project 
called JUPITER (JUvenescent Ploneeing TEchnology for Robots), which is a 
research and development program of advanced robot technology for a system 
that avoids the need for humans to work in potentially hazardous working 
environments, such as nuclear power plants, under sea, and disaster areas. 

In previous papers [1,2], the principle of the tele-existence 
display method was proposed. Its design procedure was explicitly defined. 
Experimental display hardware was made, and the feasibility of the visual 
display with a sensation of presence was demonstrated by psychophysical 
experiments using the test hardware. In the latest paper [5], a method was 
proposed to realize a mobile tele-existence system, which can be remotely 
driven with the auditory and visual sensation of presence. A prototype 
system was constructed and the feasibility of the method was evaluated. The 
effectiveness of the proposed system was evaluated by navigation 
experiments of the mobile robot through an obstructed space. Several 
display and operation methods were compared quantitatively using the time 
elapsed, smoothness of the travelled path and the number of collisions as 
the criteria for comparison. 
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In this report the concept of the tele-existence is shown, the 
principle of the tele-existence display method is explained, some of the 
prototype systems are described including an anthropomorphic robot with 
seven degrees of freedom arm, which is designed and developed as a slave 
robot for feasibility experiments of teleoperation using tele-existence 
method. 

2. Tele-existence 

In tele-existence, an autonomous anthropomorphic robot is placed at 
a remote site and an information transmission communication channel is 
established between human and robot. The operator's movements and physical 
status are sensed and transmitted to the robot via this communication 
channel. The transmitted signals override the autonomy of the robot and 
directly control the robot's motor system to reproduce the exact movements 
of the operator in its artificial eyes, neck, hands, legs, and feet. 
Information picked up by the artificial sensory organs of the robot are 
transmitted back to the operator via the communication channel to the 
operator's sensory organs. 

Take vision for example. Whichever direction the operator looks, 
the robot will look in the exact same spot. The operator will see on his 
retinae the image seen by the robot, in exactly the same manner as it would 
be seen by a human in the same position. If the operator were to bring his 
arm in front of his eyes, he would see the robot's arm being brought into 
his field of view in exactly the same relative position as his own arm. 

Thus the operator is able to maintain his or her visual sensation and 
proprioceptive sensation coherent. The operator can perform tasks via a 
robot at a distance yet maintain the same spacial relation among the 
objects, the arms and the environment as that by direct observation. 
Auditory and tactile sensations are also transmitted to the operator. 
Objects touched by the robot are also felt by the operator as tactile 
stimul i . 


Tele-existence technology also goes beyond the scope of the human 
senses. Radiation, ultra-violet rays, infrared, micro waves, ultrasonic 
waves, and ultra low frequency sound information sensed by the robot 
sensors can also be utilized to augment the human operator. For example 
infrared information picked up by the robot sensors can be converted into 
visible light on the operator's display. As the display gives a realistic 
sensation of presence, tasks can be performed in the dark yet with the 
illusion that it is light. These pieces of information can also be 
superimposed on the visual display as three dimentional superimposition. 

For example, by adding distance information at the location of an object. 

It is also possible to display human like solid model arms instead of the 
mechanical robot's arms, which enhances the operator's sensation of 
presence. 

The final version of the tele-existence system will be consisted of 
intelligent mobile robots, their supervisory subsystem, a remote-presence 
subsystem and a sensory augmentation subsystem, which allows an operator 
to use robot's ultrasonic, infrared and other, otherwise invisible, sensory 
information with the computorgraphics-generated pseudorealistic sensation 
of presence. In the remote-presence subsystem realistic visual, auditory. 
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tactile, kinesthetic and vibratory displays must be realized [1]. 

3. Principle of Tele-Existence and Means of Realization 

The basic configuration of the tele-existence system is shown in 
Fig. 1. Take vision as an example to explain the principle of the display 
which gives a sensation of presence [1]. 

The system is based on the principle that the world we see is 
reconstructed by the human brain using only two real time images on the two 
retinae of a human. What we get from the environment are only 
two-dimensional pictures on the retina changing in real time according to 
the movement of the eyeballs and the head. We reconstruct the 
three-dimensional world in the brain and project the reconstructed world to 
the real three-dimensional world [1]. 

In our new type of robotic display; 

(a) human movements including a head and/or eyeballs are precisely measured 
in real time, 

(b) robot sensors and effectors are constructed anthropomorphical ly in 
function and size, 

(c) movements of the robot sensors are controlled precisely to follow the 
human operator's movement, and 

(d) the pictures taken by the robot sensors are displayed directly to the 
human eyes in a manner which assures the the same visual space as is 
observed directly at the robot's location. 

This display enables an operator to see the robot's upper 
extremities, which are contrlled to track in real time precisely the same 
movement of the operator's, instead of his/hers at the position his/her 
upper extremities should be. 

4. Design of the Visual Display with Sensation of Presence 

Essential parameters for human three dimensional perception of an 
object are: (1) the size of the retinal image of the object, or visual 
angle, (2) convergence of the two eyes, or equivalent disparity of the two 
retinal images, and (3) accomodation of the crystalline lenses. Adding to 
the above monochromatic parameters, fidelity in color is important for a 
realistic display [5]. 

Figure 2 (a) shows a schematic diagram of the direct observation of 
an object in three dimensional space. The human observer measures the 
convergence angle ( oL ) and the size of the object on the retina (lm). Since 
the distance between the two eyes (Wm) and the distance between the 
crystalline lens and the retina (am) are known, a human observer can 
estimate the distance to the object (dobj) and the size of the object 
(lobj) as follow [5]: 

dobj = Wm/2tan( o£/2) (la) 

lobj= dobj*lm/am (lb) 

If we think of a virtual plane at a distance of dvir perpendicular 
to the direction of the head; and project the object image onto the plane 
as shown in Fig. 5, and the human observer observes the projected images by 
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using the corresponding eyes, then the observed parameters, i.e., of and 
lobj, are the same and the human observer gets the same lobj and dobj. The 
lobj and dobj can be derived by using the equivalent disparity (ed) on the 
virtual plane and the projected image size on the virtual plane (lvir) as 
follows: 


dobj = Wm*dvi r/(Wm-ed) (2a) 

lobj = dobj*lvir/dvir (2b) 

where dvir is the distance to the virtual plane. 

Figure 2(b) shows the display system which reproduces the same 
situation as the direct observation. Two TV displays and lens systems 
produce the virtual images of the size lvir on the virtual plane at the 
distance of dvir with the equivalent disparity of ed. 

Figure 2(c) shows the slave robot's camera system, where the 
distance between lenses Ws is set to be equal to Wm. The distance between 
two CCD devices (wcam) is usually ,but not necessarily, set as 
Wcam=Wdis=Ws, where Wdis is the distance between the two centers of the TV 
displays. 

Under these conditions, we define a magnification factor If = 
Idis/ls. Then by arranging am=)f*as, we have the condition of Fig. 6, which 
is the same condition as for a direct observation. Practically, am can be 
determined by measuring the size of the image on the diplay (ldis) when 
monitored through the TV camera for a known size object lobj at the known 
distance dobj as: am = p *dobj, where ^ = ldis/lobj. 

The focal length of the lens (fm) must be selected to meet the 
condition that the virtual image of the TV display is on the virtual plane. 

Ideally the distance to the virtual plane (dvir) should be 
controlled to coincide with the dobj controlling both fm and am. However, 
experiments revealed that if 200 mm <^dobj < oo , dvir can be fixed to 1000 
mm, and if 145 mm £ dobj £ 2000mm, dvir can be fixed to 500 mm. This makes 
the design and realization of the system more practical. 

If these conditions are satisfied and the cameras and the display 
system follow the head movement of the operator, the ideal condition of the 
direct observation is always maintained [5]. In order to have a wide view 
without moving the operator's head, a short focal length of the camera (fs) 
must be selected and the appropriate values for as and am must be set. 

5. Design and Control of Display Mechanisms 

5.1. Master-Slave Controlled Active Display Mechanism [2] 

Figure 3 shows the experimental hardware system for the evaluation 
study. The movement of the head of the human subject is measured in real 
time by the light weight goniometer with six degrees of freedom. Three 
translational coordinates (x,y,z) and three rotational angles (roll, pitch, 
yaw) are calculated by a microprocessor, and both the camera position/ 
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orientation and the display position/orientation are servo-controlled to 
follow the head movement. 

As the number of degrees of freedom of the allowed head movement is 
large, it is impossible to use the torque produced by the operator's head 
movement as the energy source of the movement of the display device. Even 
if the weight is removed by a counter balance mechanism, the inertia can 
not be eliminated. Therefore, it is necessary to servo-control the display 
device. 


The active display mechanism shown in Fig. 3 has five degrees of 
freedom. Each degree of freedom is actuated by a direct drive torque motor 
(Inland Rare Earth D.D. Torque Motor). The dispaly follows the goniometer's 
movement like a master-slave manipulator system with the goniometer as a 
master and the display as a slave. 

Visual displays were designed according to the procedure proposed 
in section 4 using two 1.5 inch color CRTs. The visual tele-existence 
system with these displays were experienced by several subjects. All 
subjects had an impression that this type of display produces very 
realistic feeling of remote presence. Adding to the qualitative evaluation, 
objective and quntitative experiments were also conducted [2], 

Experiments with this hardware revealed, however, that the 
position/ orientation control is not enough. Subjects usually want a 
compliant motion which follows their head movement. Therefore, force 
control based on the measurement of the head movement and/or force 
condition becomes necessary. 

5.2. Impedance Controlled Active Display Mechanism 

Figure 4 shows a general view of the impedance controlled head- 
coupled display with two degrees of freedom. It has an active power 
assistance mechanism and its impedance can be controlled by internal 
feedback loop. We used direct drive motors to attain this mechanisms, and 
the dedicated computer controls the impedance of the display mechanisms so 
that the human operator feels only quite low inertia compared with the 
physical inertia of the system. 

The dynamic equation of a system with one degree of freedom can be 
expressed as follows: 

Ktl + To = J 0 s* + Fb 0 s + Fc, (3) 

where 0 is the motor rotary angle, I is the motor current, Kt is the 
sensitivity of the motor torque, To is the torque caused by the manual 
force, J is the moment of inertia, Fb is the viscous friction coefficient, 
and Fc is Coulomb's friction torque. 

By substituting 

I = {ol J0s*+pb0s +jfFc)/Kt, ,^<1) (4) 

into equation (3), we find that 
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(5) 


To = (1-tf)J0s z + (1-^)Fb0s + (l-JT)Fc. 

which exhibits the effects attainable by multiplying the inertia force, 
viscous friction force, and Coulomb's friction force (1-ClO. (1-/5), and 
(1- )f) times as high, respectively. In other words, it is possible to 
redesign the motion equations of the system (or impedance of the system) 
into an arbitrary form through internal feedback [6,7]. An extention of 
the method to the multiple degrees of freedom system is shown in [6], 

Figure 4(a) shows the impedance controlled display using two 3 inch 
color LCDs, and Fig. 4(b) shows the anthropomorphically arranged slave 
camera system with two degrees of freedom. When operators actually wore the 
display and moved it by neck force, the reaction force caused by inertia 
appeared to be lighter, and they reported that the difference was 
particularly noticeable when the display was moved swiftly. Operators felt 
that the system is quite similar to a passive mechanism of lighter weight. 

5.3. Head Mounted Display 

A Head mounted display is also a promising design approach. The 
merit of the head mounted display is that an operator can move around quite 
easily, while that the human operator must support all the weight by 
himself becomes its demerit. Since gravitational force and the inertia of 
the system can not be compensated in this system, the design of light 
weight display is quite important. 

Figure 5 shows the head-mounted display Mk. I. It weighs 1.7 Kg, 
including a helmet ( 620 g for the display). It uses two 4 inch color 
liquid crystal TV displays (resolution: H320 x V220). Eye lenses which are 
used to attain the effect of Fig. 2(b) are mounted on a spectacles' frame. 
Lighter version of the head mounted display Mk. II has been made. Its total 
weight is 600 g. 

6. Tele-Existence Experimental Systems 
6.1. Mobile Tele-Existence System [5] 

A prototype system with fundamental mobile tele-existence functions 
has been assembled for experimentation. The system consists of an 
independent mobile robot with two TV color cameras, a remote control 
station with the visual and auditory displays with a sensation of presence, 
and a communication link between the human operator and the mobile robot. 

During routine navigation tasks, the robot travels autonomously 
using the environmental map and the environmental information gathered by 
the visual sensors (two TV cameras and an ultrasonic sensor) and internal 
sensors (two odometers on the rear wheels). 

The navigation process can be monitored by the operator. When the 
robot encounters a task which the robot is not able to manage by itself, it 
stops and asks the operator for help. At that time the operator controls 
the robot using joysticks as though he were driving that robot like an 
automobile, i.e., as if he were on board the robot at the position where 
the robot's TV cameras are located. 
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Figure 6(a) shows the head— linked display with a sensation of 
presence used in the system, while Fig. 6(b) shows the prototype 
tele-existence mobile robot constructed. 

6.2. Tele-existence Manipulation System 

Figure 7 shows a general view of the anthropomorphic slave robot 
with an operator wearing head-mounted display explained in Section 5.3. 
Figure 8 shows an operator using the impedance controlled display explained 
in Section 5.2. In the latter system electromagnetic sensor is used for the 
control of the manipulator. 

The slave robot has a three degrees of freedom neck mechanism on 
which stereo camera is mounted. The robot's structural dimensions are set 
very close to those of human's, and it is controlled to follow the human 
movement. 

The human operator's head movement is measured in real time using 
electromagnetic sensor and the slave robot's neck is controlled to follow 
the master's movement. The streo color video signals are sent to the human 
operator and displayed as a fused image, which keeps the distance, 
direction and size of the object as those of the direct observation. 

7. Tele-Existence Simulator 

Extension of the tele-existence to the artificially constructed 
environmental information has been sought, the visual tele— exi stence 
simulator has designed, pseudo-real-time binocular solid model robot 
simulator has been made, and its feasibility has been experimentally 
evaluated [8]. 

Two main situations for the simulator usages are: 

(1) To provide the operator information of the remote environment which 
human senses do not work but the robot's sensors do. For example, at night 
infrared sensor information is converted to visible light to see an object 
in the dark. It is also possible to superimpose range information gathered 
by the robot's ultrasonic and/or laser range sensors to the three 
dimensional visual display. The operator can effectively use this piece of 
information to augment human ability. 

(2) To provide totally artificial but realistic environmental information 
to the operator, e.g., realization of virtual terminal or virtual console 
for the operator [3]. The operator can enjoy variety of consoles without 
changing them physically. This can also be used for the simulation study 
for training and also for optimal parameter selection and evaluation of 
man-machine system. The usage of the system as scientific tools for the 
analysis of human visual sensation, motion control and sensor-motor 
coordination is also possible. 

As the first step toward the goal, a solid model robot manipulator 
with pseudo-real-time shading capability was constructed. By using the 
specially designed binocular optical system, three dimensional observation, 
which can exactly assign the distance and the size of the manipulator and 
an object, became possible. 
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8. Space Applications 


Wide variety of application of tele-existence technology to space 
can be conceived, including tele-observation, tele-maintenance, 
tele-construction, tele-experiment, and tele-experience. 


M . . . In Japan the Space Robot Resarch Planning Committee for the 
Ministry of International Trade and Industry has been organized. In the 
committee a space robot which replace some of the human extravehicular 
activities is being planned. Tele-existence will be intensively applied for 
the design of the space tele-robot. 


In order to apply tele-existence technology to a tele-robotic 
system whose slave robot is located far away and the time delay for 
communication can not be neglected, it is important for a slave robot to be 
autonomous. Foundamental study for the realization of tele-existence using 
autonomous robots as slaves under the circumstances is now being conducted 
in the authors' laboratory. 
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Fig. 1 Concept of the tele-existence. 




Fig. 2(a) 


a : convergence angle 
ed : equivalent disparity 

Visual parameters of direct observation. 


a * convergence angle 
ed ■ equivalent disparity 

Fig. 2(b) Parameters through the display. 
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